Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification

A deep-learning technology for knowledge transfer is necessary to advance and optimize efficient distillation. Here, we aim develop a new adversarial optimization-based method involved with layer-wise dense flow that distilled from pre-trained deep neural network (DNN). Knowledge distillation transferred another target DNN based on loss functions has multiple flow-based items are densely extracted by overlapping them enhance the existing knowledge. We propose semi-supervised learning-based of DNN. The proposed function would comprise supervised cross-entropy typical classification, an training discriminators, Euclidean distance-based in terms flow. For both DNNs considered this study, adopt residual (ResNet) architecture. methods (1) adversarial-based optimization, (2) extended scheme, (3) combined network. results show it provides higher accuracy performance improved ResNet compared prior methods.